Unified Stereo-Based 3D Head Tracking Using Online Illumination Modeling

This paper investigates the estimation of 3D head poses with a partial ellipsoid model. To cope with large out-of-plane rotations and translation in depth, we extend conventional head tracking with a single camera to a stereo-based framework. To achieve more robust motion estimation (even under time-varying lighting conditions), we incorporate illumination correction into the aforementioned framework. We approximate the face image variations due to illumination changes as a linear combination of illumination bases. Also, by computing the illumination bases online from the registered face images, after estimating the 3D head poses, user-specific illumination bases can be obtained, and therefore illumination-robust tracking without a prior learning process can be possible. Furthermore, our unified stereo-based tracking is approximated as a linear least-squares problem; a closed-form solution is then provided.